DOCUMENT RESUME 



ED 456 297 



CE 082 277 



AUTHOR 
TITLE 
PUB DATE 
NOTE 



AVAILABLE FROM 

PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Mitchell, John 

The Challenges in Developing VET Competencies in E-Commerce. 
2001-03-00 

lip.; In: Research to Reality: Putting VET Research To Work. 
Proceedings of the Australian Vocational Education and 
Training Research Association (AVETRA) Conference (4th, 
Adelaide, Australia, March 28-30, 2001); see CE 082 232. 

For full text: 

http : / /www . avetra . org . au/PAPERS %2 02 0 0 1 /mi tchel 1 . pdf . 

Reports - Evaluative (142) -- Speeches/Meeting Papers (150) 

MF01/PC01 Plus Postage. 

Adult Education; Basic Business Education; *Business; 
Business Skills; *Competence; *Computer Mediated 
Communication; Developed Nations; Educational Certificates; 
Foreign Countries; Formative Evaluation; *Job Skills; 
Postsecondary Education; Program Effectiveness; Program 
Evaluation; Secondary Education; Standard Setting; *Student 
Certification; *Technological Advancement 
Australia; *Electronic Commerce 



ABSTRACT 



A formative evaluation was begun of an innovative project 
funded by the Australian National Training Authority (ANTA) to develop 
competencies and qualifications in e-commerce. The formative evaluation was 
designed to focus on inputs, processes, and interim outputs, identifying both 
good practice and areas for improvement. Findings to date indicated 
development of competencies and qualifications in e-commerce was challenging 
developers for a number of reasons. First, there was an ongoing, 
international debate about the meaning of the term e-commerce. Second, the 
field of e- commerce was fluid, due to the continual development of new 
technologies that enable the creation of new business practices. Third, the 
field of e-commerce was in flux, evidenced by the failure of many business 
models during 2000 and the emergence of new business models. (YLB) 
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E-commerce is an important national issue, as it may provide some of the 
overarching set of business principles and practices needed to drive Australia in the 
global, networked economy. The training needs for e-commerce may become 
extensive, so it is imperative that competencies developed for e-commerce are valid 
for industry and acceptable to clients. 



This paper reports on the evaluation in 2000-2001 of an innovative project funded by 
the Australian National Training Authority (ANT A) to develop competencies and 
qualifications in e-commerce. The ANTA E-commerce Initiative project is a response 
to E-competent Australia (Mitchell 2000), which argued that it was both possible and 
important to identify competencies in e-commerce. E-competent Australia also noted 
the huge demand for VET courses in e-commerce and the need to design 
competencies carefully. 



Two industry training advisory boards (ITABs) - the information technology and 
telecommunications (IT&T) ITAB and the business services (BST) ITAB - were 
commissioned in 2000 to develop approximately 100 competencies in e-commerce. A 
further four ITABs will commence the development of around 30 competencies in 
mid-2001. A project manager external to ANTA was appointed to oversee the project. 



The evaluation of the ANTA E-commerce Initiative Project commenced in September 
2000 and will conclude in early October 2001. This interim report on the evaluation 
highlights the complexities faced by VET course designers working on the leading 
edge. 

The author is undertaking a Doctorate in Education in the Department of Education 
at Deakin University (having commenced in 1997), focusing on e-commerce in the 
VET sector, particularly in relation to flexible and online delivery. The author brings 
his learning from this earlier research into e-commerce to this evaluation. 



Methodologies for the formative evaluation 

The purpose of the evaluation of the ANTA E-commerce Initiative is to assess both 
the efficiency and effectiveness of the project. The formative evaluation focuses on 
inputs, processes and interim outputs, identifying both good practice and areas for 
improvement. The summative evaluation will clarify how effectively the project's 
^ outcomes achieve the project's stated objectives. This framework of efficiency and 

^ effectiveness is a modification of one proposed by the Department of Finance (1994) 
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The formative evaluation of the project measures the efficiency of the project, and the 
summative evaluation measures the effectiveness of the project. The formative 
evaluation is concerned with: 

• the inputs, such as the resources used in the project (eg the staff supporting 
the project); 

• the processes (activities, strategies, operations) by which competencies are 
developed (taking into account the particular contexts for each Training 
Package developer, eg the internal evaluation strategies used by the two 
ITABs); and 

• the outputs, ie the deliverables provided by the project (eg the number of 
competencies developed) over which the developers have direct control. 

Efficiency is about improving processes and doing it better. Formative evaluations of 
efficiency are undertaken regularly during the project, to determine how inputs, 
processes and outputs may be improved. The formative evaluation provides an 
opportunity to share best practices, and these lessons are discussed with the Project 
Manager and each ITAB, particularly at regular workshops held with the ITABs. 

Table 1 relates the methodologies used in the formative evaluation of the objectives. 



Table 1: Objectives and methodologies for formative evaluation 



Description of 
evaluation of efficiency 


• Evaluation of the project conduction 


Objectives of 
evaluation of efficiency 


1. To make recommendations about improvements to the 
conduct of the projects 

2. To provide early warning advice on aspects of the project 
that may emerge as problem areas 

3. To provide ongoing analysis, identifying significant 
issues that promote or inhibit the success of the project 


Focus of evaluators' 
attention (inputs, 
processes, outputs) 


1. Project resourcing 

2. Project documentation 

3. Project activities, strategies, operations 
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Qualitative 

methodologies 


1. Routine monitoring of inputs, processes and outputs 

2. SWOT analysis of individual training package developer 
project 

3. Case studies of field tests. 

4. Interviews with key stakeholders 


Quantitative 

methodologies 


1. Analysis of documentation of actual numbers of 
competencies and qualifications developed 

2. Questionnaire (for stakeholders) 


Audiences 


• Training Package developers 

• Project Manager 

• Steering Committee 

• Expert Steering Committee 

• Other ITABs 



Methodologies for the summative evaluation 

Evaluations of effectiveness are also called summative evaluations. Effectiveness is 
defined as the extent to which a program's expected and unexpected outcomes 
achieve its stated objectives. Evaluating effectiveness is about accountability: it is 
about the setting of objectives, producing and reporting on outcomes, and the 
consequences of getting things right or wrong. 

The objectives of the ANTA E-commerce Initiative project are to extend the endorsed 
and support materials of existing Training Packages with new and enhanced 
competency standards and qualifications for e-commerce as part of the ANTA e- 
commerce initiative for the National Training Framework. Together with a minor 
enhancement proposal for modifications to the current standards and qualifications, 
the outcomes of this project will provide a contemporary range of competency 
standards and qualifications for application in industry. Working to the E-commerce 
Project Manager, for each ITAB, this will involve endorsed components and support 
materials. 



Endorsed components 



• drafting and validating new and revised competency standards for inclusion 
in Training Packages that fit with existing assessment guidelines; 

• packaging and aligning the new and revised competency standards into new 
e-commerce qualifications that fit with existing assessment guidelines; and 

• revising existing Training Packages to include the e-commerce standards and 
qualifications. 

Support materials 

• identifying available learning resources that would support delivery of the 
endorsed components; 

• developing validated support materials for learners, facilitators and RTOs 
that effectively contribute to learning and assessment processes; and 

• in the case of the IT&T ITAB, revising existing materials (Resources Data 
Base, Case Study Maker, Traineeship Generators and Professional 
Development products) to incorporate the e-commerce standards and 
qualifications. 

Table 2 relates the methodologies used in the summative evaluation of the project 
objectives. 



Table 2: Objective and methodologies for summative evaluation 



Description of 
evaluation of 
effectiveness 


Objective to be evaluated 


Qualitative 

methodologies 


Quantitative 

methodologies 


Evaluation of the 
outcomes of the 
sub-projects 


To extend the endorsed and support 
materials of existing Training Packages 
with new and enhanced competency 
standards and qualifications for e- 
commerce as part of the ANTA E- 
Commerce Initiative for the National 
Training Framework. 


1. Case studies 

2. Observations 

3. Interviews 


1 . Review of 
numbers of 
competencies 

2. Review of 
numbers of 
qualifications 



The results of the summative evaluation of the ANTA E-commerce Initiative will 
inform a second 12 months of the Initiative, if a second stage is funded. 



Findings 

This section reports on the general findings from the evaluation for the period 
September 2000 to March 2001. The section demonstrates that the major challenge to 
developers of competencies in e-commerce is that the field of e-commerce is 
changing rapidly. 
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Changing business models 

Business models involving e-commerce changed during 2000, producing many 
challenges for developers of e-commerce competencies. Changes to business models 
were affected by developments such as: 

• In April-March 2000, the sharp fall in the stockmarket value of many 
companies that solely used online technologies: the 'dotcoms'. 

• The rise in the popularity of business-to-business e-commerce, particularly 
trade exchanges, epitomised in Australia by the Cor Procure group of 14 
different companies (AMP, ANZ, Australia Post, Amcor, BHP, Coca-Cola 
Amatil, ColesMyer, Fosters, Goodman Fielder, Orica, Pacific Dunlop, Qantas, 
Telstra and Westpac). 

• The emergence of new business models during 2000, such as ColesMyer' s 
ColesOnline, which is integrating bricks and mortar business with its online 
operations. (In the final week of November 2000, four of the top five retailing 
web sites in Australia were owned by conventional department store retailers 
- Myers, Kmart and Target (owned by ColesMyer) and David Jones were the 
first, third, fourth and fifth most popular sites. The second most popular site, 
dstore, is now owned by retailer Harris Scarfe. 

• Changes to the business models of companies such as Wine Planet that was 
previously only a dotcom operation. Wine Planet is now developing 
conventional retail outlets. 

• The reduction in value in late 2000 of some prominent dotcoms that survived 
the March- April 2000 sharemarket fall, but experienced difficulty in late 2000, 
such as the online share trading company Etrade and the world's largest 
portal Yahoo! 

• Mergers between dotcoms and conventional businesses, eg Harris Scarfe's 
purchase of dstore and Woolworths' acquisition of shares in 
Greengrocer.com. 

• Mergers between former rivals in the dotcom domain, in areas such as online 
auctions. 

• The surge in the number of educational organisations, mostly in the USA, 
that only operate with a web interface with the student. 

• The rise of mobile telecommunication technology, such as text messaging via 
mobile phones, that shows that e-commerce does not always require 
computer networks. 

An interesting story to emerge from the 'dotbomb' year 2000 is the success of online 
auction site e-Bay. In the third quarter of 2000 alone, e-Bay hosted 68.5 million 
auctions, facilitating the exchange of $US1.4 billion in merchandise ( Sydney Morning 
Herald, Jan 2001). Its business model avoids two of the flaws of other dotcoms: it does 
not rely on banner ads for revenue (Yahoo's problem) and it does not need to 
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warehouse goods (Amazon.com' s problem). EBay's business model works: it is an 
online exchange connecting buyers and sellers and collects a fee for each transaction 
{Sydney Morning Herald , Jan 2001). 

Ongoing debates about the nature of E-commerce 

As of March 2001, there are ongoing debates about a number of key aspects of e- 
commerce, increasing the difficulty of developing competencies in e-commerce, 
namely: 

• whether the distinction between 'old' and 'new economy' is valid or 
worthwhile; 

• whether e-commerce will worsen the 'digital divide': the gap between those 
who use online communication and those who don't; 

• whether business-to-business networks will have negative effects on small 
providers; 

• whether all business-to-business networks will deliver the benefits sought by 
their owners (Forrester Research predicted in August 2000 that of the 1,000 
online marketplaces in the USA, fewer than 2000 will exist in 2003 - see BRW 
15 Dec 2000); 

• whether dotcoms will be able to overcome an apparent weakness in 
'fulfilment' of online orders; 

• whether the security and privacy issues will be solved and users will be 
convinced that they can safely provide their credit card details to online 
companies; 

• whether vertical business-to-business networks that link together companies 
in one specific industry are more likely to succeed than horizontal networks 
that link disparate organisations from many industries. 

These ongoing debates will ensure that the definition of e-commerce will continue to 
be challenged. 

Continual development of new technologies 

E-commerce will change in 2001 due to the implementation of new versions of 
previous technology and the availability of wholly new technologies, which means 
that new competencies in e-commerce will continue to be needed. For example, a 
report in the Sydney Morning Herald on 14 October 2000, drawing on research from 
Forrester Research, Gartner Group, Dataquest and Yankee Group, identified the 
following information technology and business trends. 

• Spending on e-commerce software is expected to climb from $US3.1 billion in 
1999 to $US14.5 billion in the US alone by 2003. 'As B2B and B2C markets 
become more competitive, vendors will need increasingly sophisticated 
software'. 



• Due to the shortage in IT skills within organisations, there will be an 
increasing demand for providers of corporate IT services. 

• The worldwide Application Service Provider (ASP) market will grow from 
$US3.6 billion in 2000 to $US25.3 billion in 2004, as software transforms from 
a product to a service. However, 60% of ASPs are expected to fail by 2001. 
'This will have a devastating effect on businesses that have outsourced their 
data services to an insolvent ASF. 

• The B2C will continue to be the USA leader, with online retail sales expected 
to reach $US10 billion in the US in Christmas 2000; double the total in 1999. 
By 2003, US consumers and businesses will spend $US2 trillion over the Net, 
but retailing will make up just $US144 billion of the total. 

• The B2B e-commerce activity will continue to dominate, growing quickly 
from $US145 billion worldwide in 1999 to $US7.3 billion by 2004. By 2004, 
B2B e-commerce will represent 7% of total global sales transactions. 

• Broadband technologies will continue to be provided. In the Asia-Pacific 
there will be 11.3 million residential broadband subscribers by 2003, mostly 
using cable modems and ASDL (high-speed access over normal telephone 
lines). The roll out is happening more quickly in the USA than in Australia or 
Europe: in the USA 3.3 million homes had broadband access in 2001 and this 
is expected to grow to 16.6 million homes by 2004. 

• Telecommunications infrastructure, particularly the provision of fibre optic 
technology, is expected to grow exponentially, to satisfy the rapidly 
increasing demand for data traffic. Wireless technology will become more 
common, enhanced by new and faster mobile phone networks. Wireless 
technology is expected to move through a number of different generations in 
the next few years. 

• As sales of PCs slow, the focus of chip makers will shift from PCs, to mobile 
phones and other wireless devices, providing increased memory and new 
consumer products. The global market for memory chips will grow from 
$US222 billion in 2001 to $US320 billion by 2004. 

Debates about the terminology 

The main difficulty for developers of e-commerce competencies is that the definition 
of e-commerce and its popular alternative e-business continues to be debated. To 
define e-commerce, firstly we need to clarify the changing definitions in 1998-1999, 
as discussed in E-competent Australia (ANTA 2000). The definitions of e-commerce 
shifted in 1998-1999 in two major respects: 

• It was common during 1998 to see e-commerce defined as 'financial 
transactions over the Nef, but the trend by late 1999 was to define e- 
commerce as any business communication involving electronic 
communication, including internal communication. 



• In 1998, e-commerce was often defined as the technical event of an electronic 
communication, but the trend by late 1999 was to see e-commerce as an 
approach to business, with the technology as the enabler. 

These changes to the definition in 1999 were modelled by the National Office for the 
Information Economy (NOIE). For instance, the following definition was provided 
by NOIE in April 1999 in Australia's e-commerce report card : 

E-commerce is defined as every type of business transaction in which the 
participants (i.e. suppliers, end users etc.) prepare or transact business or 
conduct their trade in goods or services electronically, (p 3) 

While e-commerce is dominated by online technologies, the scope of e-commerce 
covers 'all forms of electronic processes': 

Online technologies are the most significant facets of e-commerce and 
include Internet retailing, Electronic Data Interchange, Internet banking, 
electronic settlements and browsing and selection of products and 
services over the Internet, (p 3) 

This 1999 definition is much more substantial than previous definitions of e- 
commerce as buying and selling over the net. 

A fuller definition of e-commerce was provided in NOIE's October 1999 report, E- 
Australia.co7ii.au. The October 1999 definition widens the scope of e-commerce, and 
clarifies that e-commerce is not just about buying and selling goods; it is also about 
inter- company and intra- company activities: 

In e-commerce, business is communicated and transacted over networks 
and through computer systems. The most restrictive definition limits e- 
commerce to buying and selling goods and services, and transferring 
funds through digital communications. However, e-commerce also may 
include all inter-company and intra-company functions (such as 
marketing, finance, manufacturing, selling, and negotiation) that enable 
commerce and use electronic mail, EDI, file transfer, facsimile, video- 
conferencing, workflow, or interaction with a remote computer. E- 
commerce also includes buying and selling over the World Wide Web 
and the Internet, transferring electronic funds, using smart cards and 
digital cash, and doing business over digital networks, (p 60) 

In E-competent Australia (Mitchell 2000), this October 1999 definition was 
recommended as the main reference point for the ANTA E-commerce Initiative, 
taking into account the fact that the nature and definition of e-commerce may change 
in future, due to new developments in business and technology. 

In E-competent Australia (Mitchell 2000), e-business was taken to mean an individual 
company, enterprise or organisation or business unit that uses e-commerce, eg the 
online travel agency travel.com.au could be called an e-business. However, in 
defining e-commerce as 'doing business electronically' (Timmers 1999), E-competent 
Australia (Mitchell 2000) was suggesting that e-commerce is an overarching set of 
business principles behind new ways of doing business electronically. 

However, the use of the terms e-commerce and e-business have continued to change 
since the publication of E-competent Australia (2000) and different interpretations of 
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the terms e-commerce and e-business may continue in Australia for some time, for 
the following reasons. Firstly, some people see e-business as interchangeable with 
the term e-commerce. Secondly, the popular media likes to use e-business instead of 
e-commerce and to place the prefix e- before many words, particularly in newspaper 
headlines. Thirdly, academics are divided in their definitions of the two terms, with 
some seeing e-commerce as the overarching concept and others seeing e-commerce 
narrowly as buying and selling electronically. While we can expect that e-commerce 
and e-business will continue to be defined differently, present indications are that the 
term e-business may emerge as the most popular one in daily use. 

The ANTA e-commerce project commenced by using the term e-commerce as the 
basic reference point. Business Services Training clearly prefers the term e-business, 
which is understandable, given its close relations with the business community, 
while IT&T ITAB generally uses the term e-commerce and sometimes e-business for 
the project. In late March 2001, the evaluator and the two ITABs agreed that, in the 
majority of cases, the term e-business would be used in relation to the competencies 
being developed. However, a glossary will be developed, providing definitions of e- 
business and e-commerce. The reasons for this decision included the strong desire of 
the Business Services ITAB to use e-business, and the value of not causing concern 
for trainers by using e-business for one set of competencies and e-commerce and e- 
business for the other set. 

Discussion 

To date, the formative evaluation of the ANTA project shows that the development 
of competencies and qualifications in e-commerce is challenging developers for a 
number of reasons. Firstly, there is an ongoing, international debate about the 
meaning of the term e-commerce. Secondly, the field of e-commerce is fluid, due to 
the continual development of new technologies that enable the creation of new 
business processes. Thirdly, the field of e-commerce is in flux, evidenced by the 
failure of many business models during 2000 and the emergence of new business 
models. 

The definition of e-commerce will continue to be debated, affected by the 
development of new technologies, new business thinking and changing business 
practices. While debate will continue about the best business models and practices, 
there will be no preventing the inevitable increase in e-commerce activity, leading to 
the need for additional training. 
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